A Neural Stiefel Learning based on Geodesics Revisited

In this paper we present an unsupervised learning algorithm of neural networks with p inputs and m outputs whose weight vectors have orthonormal constraints. In this setting the learning algorithm can be regarded as optimization posed on the Stiefel manifold, and we generalize the natural gradient method to this case based on geodesics. By exploiting its geometric property as a quotient space: homogeneous space, the previous result [11] for the case of the orthogonal group can be used to derive the algorithm. Relevant as well as possible applications of the geometry of homogeneous spaces are also suggested. Key-Words: natural gradient method, Riemannian metric, geodesic, Lie group, Stiefel manifold, Riemannian submersion, shape theory, unsupervised neural learning